CHIP-0034: keccak and base64 operators #116

danieljperry · 2024-04-26T04:15:32Z

No description provided.

freddiecoleman · 2024-04-26T16:40:41Z

Thanks, @danieljperry ! This CHIP has been assigned CHIP-34 and is now a Draft. Feel free to leave a review here, or to discuss it in the #chips channel of our Discord.

greimela · 2024-04-27T08:51:33Z

Great addition!

Regarding passkey support: the passkey standard uses the urlencoded version of base64, without padding. So those two aspects could be configurable via arguments, or via separate operators.

CHIPs/chip-0034.md

Rigidity

Looks good so far

CHIPs/chip-0034.md

danieljperry · 2024-05-01T01:58:42Z

Thanks for the comments! I have updated the CHIP accordingly.

arvidn · 2024-05-01T13:27:01Z

CHIPs/chip-0034.md


 Note that these operators could have other use cases not covered in this CHIP.

 ## Backwards Compatibility

 A few notes regarding this CHIP's compatibility with the current implementation of CLVM:
 * The CLVM operators to be added are backwards compatible -- any calls that succeed after the CHIP has been implemented also would have succeeded beforehand.
-* The CLVM operators to be added are not forward compatible -- some calls that succeed before the CHIP has been implemented will no longer succeed afterward.
+* The CLVM operators to be added are not forward compatible -- calling an undefined operator will result in a successful no-op, so some calls that succeed before the CHIP has been implemented will no longer succeed afterward.


I'm not quite following what this bullet intends to communicate. all 3 new operators return some value (a hash, a binary blob or a base64 encoded string). This is fundamentally the compatibility-breaking feature. All unknown ops are allowed in consensus mode, but they return NIL. These new operators have return values, therefore it would be a hard fork to add them, had it not been for the softfork operator.

arvidn · 2024-05-01T14:01:13Z

CHIPs/chip-0034.md

@@ -36,23 +36,23 @@ CLVM currently includes an atomic operator called [`sha256`](https://chialisp.co

 This CHIP will add an atomic operator to the CLVM called `keccak256`, which will calculate and return the keccak-256 hash of the input atom(s). The primary reason to add this operator is to support Ethereum addresses, which also rely on keccak-256.

-This CHIP will also add operators to the CLVM called `base64_encode` and `base64_decode`, which will calculate and return the Base64 encoding/decoding of the input atoms(s). The primary reason to add the `base64_encode` operator is to support passkeys that rely Base64 encoding. `base64_decode` is being added for completeness.
+This CHIP will also add operators to the CLVM called `base64_encode` and `base64_decode`, which will calculate and return the Base64 encoding/decoding of the input atoms(s). The primary reason to add the `base64_encode` operator is to support passkeys that rely on Base64 encoding. `base64_decode` is being added for completeness.


Assuming I found the right specification for passkeys, could we have references:
https://passkeys.dev/docs/reference/specs/
https://www.w3.org/TR/webauthn-2/

arvidn · 2024-05-01T14:07:56Z

CHIPs/chip-0034.md

@@ -62,6 +62,10 @@ While the Keccak-256 and Base64 operators are not related, they each require sof

 Each of the new operators will incur a CLVM cost, as detailed below. If this CHIP is adopted, the new operators will be optional when designing Chia coins.

+The Base64 alphabet used in this CHIP is specified in [Section 4 of the RFC 4648 Standard](https://www.rfc-editor.org/rfc/rfc4648.html#section-4).


When following the passkey specification to webauthn-2 ( https://www.w3.org/TR/webauthn-2/ ), it clearly specifies the encoding as base64url, which is not section-4 that you reference above. It's section-5.

"Base 64 Encoding with URL and Filename Safe Alphabet"

The RFC calls this encoding base64url. Perhaps we should consider naming the functions this as well.

arvidn · 2024-05-01T14:10:19Z

CHIPs/chip-0034.md

@@ -62,6 +62,10 @@ While the Keccak-256 and Base64 operators are not related, they each require sof

 Each of the new operators will incur a CLVM cost, as detailed below. If this CHIP is adopted, the new operators will be optional when designing Chia coins.

+The Base64 alphabet used in this CHIP is specified in [Section 4 of the RFC 4648 Standard](https://www.rfc-editor.org/rfc/rfc4648.html#section-4).
+
+Each Base64 character is encoded with six bits, and decoded to eight bits. Therefore, the ending of some Base64 output strings [require padding](https://en.wikipedia.org/wiki/Base64#Output_padding). The padding used in this CHIP is `=`, which is also specified in the RFC 4648 Standard.


the webauth-2 also specifies that padding characters are omitted.

From the specification:

The term Base64url Encoding refers to the base64 encoding using the URL- and filename-safe character set defined in Section 5 of [RFC4648], with all trailing '=' characters omitted (as permitted by Section 3.2) and without the inclusion of any line breaks, whitespace, or other additional characters.

CHIPs/chip-0034.md

arvidn · 2024-05-01T14:16:51Z

CHIPs/chip-0034.md

+* If zero arguments, the result will be the Base64 encoding of an empty string (`NIL`), in other words, the result will be `NIL`
+* If one or more arguments:
+  1. Each argument (if more than one) will be concatenated
+  2. The Base64 encoding (following [Section 4 of the RFC 4648 Standard](https://www.rfc-editor.org/rfc/rfc4648.html#section-4), and padded with `=` when necessary) of the resulting string will be returned


this should refer to section-5 and it should state that no padding is produced

arvidn · 2024-05-01T14:17:51Z

CHIPs/chip-0034.md

@@ -98,14 +104,16 @@ CLVM Cost: `[todo]` base, `[todo]` per argument, `[todo]` per byte

 Opcode: [todo]

-Functionality: Calculate and return the Base64 decoding of the input atom(s)
+Functionality: Calculate and return the Base64 decoding of the input atom


I think it's more straight-forward to just say "return the decoded form of the base64 input string".

arvidn · 2024-05-01T14:18:51Z

CHIPs/chip-0034.md

-* If two or more arguments, each argument will be concatenated, and the Base64 decoding of the result will be returned
+* If more than one argument, an exception will be raised
+
+Note that the argument must be in Base64 format, encoded according to [Section 4 of the RFC 4648 Standard](https://www.rfc-editor.org/rfc/rfc4648.html#section-4). It must be padded with `=`, also according to the RFC 4648 Standard. If the argument is not encoded in the proper format, or if it is not properly padded, an exception will be raised.


Same thing here. section-5, padding is disallowed, any characters not in the encoding alphabet are disallowed. e.g. new lines

arvidn · 2024-05-01T14:20:08Z

CHIPs/chip-0034.md


 Arguments:
-* If zero arguments, the result will be a successful no-op
+* If zero arguments, the result will be an empty string (`NIL`)


it's not clear to me that we would want to support 0 arguments. Typically we only do that for operators that take a variable number of parameters

danieljperry · 2024-05-02T08:11:47Z

Thanks for the reviews, I believe I have addressed all of the concerns raised.

Yakuhito · 2024-05-03T06:50:46Z

CHIPs/chip-0034.md

+Throughout this document, we'll use the following terms:
+* **Chialisp** - The high-level [programming language](https://chialisp.com/) from which Chia coins are constructed
+* **CLVM** - [Chialisp Virtual Machine](https://chialisp.com/clvm), where the bytecode from compiled Chialisp is executed. Also commonly refers to the compiled bytecode itself
+* **Keccak-256** - The same encryption standard Ethereum uses. A few notes on this standard:


The same hashing standard Ethereum uses.

danieljperry · 2024-05-07T03:14:22Z

We will hold a public zoom call to talk about this CHIP on May 20 at the following time:
7 AM PDT / 2 PM UTC / 10 PM China
See the #chips channel of our public Discord for more info.

richardkiss · 2024-05-15T21:12:43Z

CHIPs/chip-0034.md

+
+Usage: `(base64url_decode A)`
+
+CLVM Cost: `[todo]` base, `[todo]` per argument, `[todo]` per byte


What's the motivation for base64 encoding & decoding? I understand it's something to do with ETH. Where can I find more details?

Including base64 data instead of the corresponding binary data on chain seems like a mistake, since reading base64 data would come from a computer before being dumped on chain, and there are plenty of opportunities for the reading client to decode before dumping on chain. Maybe ETH does this, due to some bad design somewhere. But I worry we're proposing changes to clvm to make simpler interoperability with bad design on ETH or elsewhere, which just encourages people to repeat those design errors on chia.

base64 is in support of passkey support.

If a signature must be validated against a base64 message, but you need to validate the message matches something within the puzzle, you have two options:

Give the puzzle the encoded base64 message and decode it, then check it matches

Give the puzzle the original message, check it matches, then base64 encode it before validating the signature

Both of these require base64 opcodes

Keccak is for Ethereum address compatibility - it allows an application to construct a Chia puzzle that is only spendable by whoever owns the private key corresponding to the Ethereum address. Which means you can send XCH to an Ethereum address with a little bit of wrapping behind the scenes

And base64 allows you to have a puzzle that can only be spent by including a signature created with a passkey (meaning you don't need to have access to the original secret key, just the signer)

It doesn’t have anything to do with ETH.

When passkeys are used via the Web Authentication API, they always encode the challenge (message) with base64 before signing it.

That’s why we need to encode the message in the puzzle too, so we can validate the passkey signature.

@greimela Appreciate the response. Do you have a reference where I can learn more about this? It seems surprising to me that they couldn't sign binary data directly and I'm wondering what possible motivation there could be.

I posted some links here: #116 (comment) and here: #116 (comment)

Yes, thanks, those are the ones!

I've looked at this more today and I think I understand this better now. WebAuthn apis sign JSON messages of a certain format, ie. certain keys must be present and must have specific formats, so that you don't sign arbitrary messages with arbitrary keys. For details, see https://www.w3.org/TR/webauthn-2/#client-data

So the thing that's signed is a JSON blob which includes a challenge: key, which is server-determined. In the prototype, we hack it to be a hash of the target conditions for the coin. Validating requires checking the signature, which is against the entire JSON blob, and then extracting the portion of the JSON blob (via an offset hint) that points to the base64 data, and comparing it against the delegated puzzle.

Yakuhito · 2024-05-20T09:30:38Z

Hi! Wanted to put this here. This CHIP could potentially allow Chia to integrate with any wallets that support ETH - including hardware wallets - by 'impersonating' a custom EVM chain during signing. From my (very brief) research, I think RLP encoding should be looked into - again, this is just a raw idea.

EDIT: As discussed in the call, because this CHIP is so far along & a new operator is needed, someone will need to create a new CHIP for the operator(s) to make it possible.

EDIT 2: After a tweet exchange, it now seems possible to bypass the RLP encoding part by using EIP-712 signatures. This ChIP would then be sufficient to add Ethereum wallet support (including hardware).

danieljperry · 2024-05-21T01:50:55Z

CHIP-34 is now in Review. Please review as time permits. We seem to have consensus on adding the three operators from this CHIP, if there are no objections from the community, we will move it to Last Call soon.

danieljperry · 2024-05-21T03:44:21Z

In case you missed it, here is our discussion of this CHIP:
https://youtu.be/hFwPBfM4s8c

danieljperry · 2024-06-06T06:25:13Z

We have decided to withdraw CHIP-34 from consideration. The reasons for this are threefold:

There isn't sufficient demand for the keccak256 operator to justify forking the consensus in order to implement it.
For base64 encoding, we are able to do this without adding a new operator. While it is true that prior to the activation of the 2.1 hard fork, the operator from this CHIP would have been significantly less costly than the non-CHIP method, this is a moot point because this CHIP would not have activated prior to the 2.1 fork's activation.

After the 2.1 fork takes full effect, block compression will be enabled. At this point, the operator from this CHIP would only have been about 2.3% more efficient in CLVM cost versus the non-CHIP version of base64 encoding. We deemed that this saving would not justify forking the consensus.

For base64 decoding, this was only ever added for completeness. There is not currently a use case for it.

If we ever do decide to implement these operators in the future, the source material will still be available for a new CHIP.

danieljperry and others added 2 commits April 26, 2024 12:14

Open CHIP for keccak and base64 operators

eb6538a

set CHIP number 0034 and status Draft

161aa89

freddiecoleman changed the title ~~Open CHIP for keccak and base64 operators~~ CHIP-0034: keccak and base64 operators Apr 26, 2024

arvidn reviewed Apr 29, 2024

View reviewed changes

Rigidity reviewed Apr 30, 2024

View reviewed changes

CHIPs/chip-0034.md Outdated Show resolved Hide resolved

CHIPs/chip-0034.md Outdated Show resolved Hide resolved

CHIPs/chip-0034.md Outdated Show resolved Hide resolved

Clarify operator functionality

16f2987

arvidn reviewed May 1, 2024

View reviewed changes

Clarify keccak and base64url standards

6caa516

Yakuhito reviewed May 3, 2024

View reviewed changes

Change encryption to hashing

6c18ba3

danieljperry added 2 commits May 8, 2024 14:06

Add softfork usage details

d33ed22

Add reference implementation PR

8fabb1d

arvidn mentioned this pull request May 15, 2024

CHIA-255: implement CHIP-34 Chia-Network/clvm_rs#403

Closed

richardkiss reviewed May 15, 2024

View reviewed changes

Add costs and move to Review

7b68c9d

Add link to w3c API for base64url encoding

6c5abef

danieljperry added 2 commits May 21, 2024 12:35

Add soft fork activation height

efd3f80

Withdraw CHIP-34

83c3a3f

danieljperry closed this Jun 6, 2024

danieljperry mentioned this pull request Oct 25, 2024

CHIP-0036: keccak256 CLVM operator #131

Open

Rigidity mentioned this pull request Oct 31, 2024

Implement keccak256 operator and softfork Chia-Network/clvm_rs#489

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CHIP-0034: keccak and base64 operators #116

CHIP-0034: keccak and base64 operators #116

danieljperry commented Apr 26, 2024

freddiecoleman commented Apr 26, 2024

greimela commented Apr 27, 2024

Rigidity left a comment

danieljperry commented May 1, 2024

arvidn May 1, 2024

arvidn May 1, 2024

arvidn May 1, 2024

arvidn May 1, 2024

arvidn May 1, 2024

arvidn May 1, 2024

arvidn May 1, 2024

arvidn May 1, 2024

danieljperry commented May 2, 2024

Yakuhito May 3, 2024

danieljperry commented May 7, 2024

richardkiss May 15, 2024

hoffmang9 May 15, 2024

Rigidity May 15, 2024

Rigidity May 15, 2024

Rigidity May 15, 2024

greimela May 16, 2024

richardkiss May 17, 2024

arvidn May 17, 2024

greimela May 18, 2024

richardkiss May 22, 2024

Yakuhito commented May 20, 2024 •

edited

Loading

danieljperry commented May 21, 2024

danieljperry commented May 21, 2024

danieljperry commented Jun 6, 2024

		@@ -62,6 +62,10 @@ While the Keccak-256 and Base64 operators are not related, they each require sof

		Each of the new operators will incur a CLVM cost, as detailed below. If this CHIP is adopted, the new operators will be optional when designing Chia coins.

		The Base64 alphabet used in this CHIP is specified in [Section 4 of the RFC 4648 Standard](https://www.rfc-editor.org/rfc/rfc4648.html#section-4).


		Usage: `(base64url_decode A)`

		CLVM Cost: `[todo]` base, `[todo]` per argument, `[todo]` per byte

CHIP-0034: keccak and base64 operators #116

CHIP-0034: keccak and base64 operators #116

Conversation

danieljperry commented Apr 26, 2024

freddiecoleman commented Apr 26, 2024

greimela commented Apr 27, 2024

Rigidity left a comment

Choose a reason for hiding this comment

danieljperry commented May 1, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danieljperry commented May 2, 2024

Choose a reason for hiding this comment

danieljperry commented May 7, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Yakuhito commented May 20, 2024 • edited Loading

danieljperry commented May 21, 2024

danieljperry commented May 21, 2024

danieljperry commented Jun 6, 2024

Yakuhito commented May 20, 2024 •

edited

Loading